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Abstract. Game theory has been one of the most successful quanti- 
tative concepts to describe social interactions, their strategical aspects, 
and outcomes. Among the payoff matrix quantifying the result of a so- 
cial interaction, the interaction conditions have been varied, such as the 
number of repeated interactions, the number of interaction partners, the 
possibility to punish defective behavior etc. While an extension to spa- 
tial interactions has been considered early on such as in the "game of 
life", recent studies have focussed on effects of the structure of social 
interaction networks. 

However, the possibility of individuals to move and, thereby, evade ar- 
eas with a high level of defection, and to seek areas with a high level 
of cooperation, has not been fully explored so far. This contribution 
presents a model combining game theoretical interactions with success- 
driven motion in space, and studies the consequences that this may have 
for the degree of cooperation and the spatio-temporal dynamics in the 
population. It is demonstrated that the combination of game theoreti- 
cal interactions with motion gives rise to many self-organized behavioral 
patterns on an aggregate level, which can explain a variety of empirically 
observed social behaviors. 

1 Introduction 

Macroscopic outcomes in a social system resulting from interactions be- 
tween individuals can be quite different from anyone's intent. For instance, 
empirical investigations have shown that most colored people prefer 
multi-racial neighborhoods, and many white people find a certain fraction 
of other races in their neighborhood acceptable. So one could think that 
integrated neighborhoods should be widely observed, but empirically this 
is not true. One rather finds segregated neighborhoods, i.e. separate urban 
quarters, which also applies to people with different social and economic 
backgrounds. 

This problem is scientifically addressed by mainly two streams of seg- 
regation theory [2]: the urban ecological "social distance" tradition in 
sociology j3|4] and the "individual preferences" tradition in economics 
|5|6j . The main idea of "social distance" theory is that the differences in 
culture and interests between social groups are reflected by a separation 
of their residential areas. 



Yet, the role of social distance and individual preferences is questioned 
by studies of the American urban housing market, which suggest that 
racial discrimination and prejudices are the primary factors of residential 
segregation and concentration of poverty [7]. There are three stages in 
a housing market transaction: first, information about available housing 
units, second, terms and conditions of sales and financing assistance, and 
third, the access to units other than the advertised unit [8]. In each stage, 
the housing agent may behave in a discriminatory way, e.g. withhold in- 
formation from customers and discourage them. Therefore, the access of 
minority customers to housing is severely constrained, while the theory 
of preference-based dynamics assumes that people can relocate freely ac- 
cording to their own preferences, which fails to reflect the real relocation 
dynamics. 

Recent studies [2 9 point out that the effect of discrimination in 
racial residential segregation was important in the past, but nowadays, 
the nature and magnitude of housing discrimination has changed. Mi- 
nority households who seek to move into integrated or predominantly 
white areas usually will be able to do so. But the integration of ethnic 
groups is not widely observed and quite unstable. This is partly because, 
when people move to a neighborhood, where they constitute an ethnic mi- 
nority, the previous inhabitants may choose to leave, while some people 
may be reluctant to enter integrated neighborhoods in which minorities 
are increasing in number, e.g. because of a decrease in the housing prices. 
Such migration dynamics, based on seeking preferred neighborhoods, does 
not necessarily presuppose the discrimination of other people, as we will 
shortly see. 

Assuming that individuals have just a slight preference for neighbor- 
hoods, in which the same ethnic background prevails, Schelling has re- 
produced residential segregation by a simple model |5|6] . Imagine that 
two groups of people are distributed over a one-dimensional lattice, and 
assume that everyone defines his/her relevant neighborhood by the four 
nearest neighbors on either side of him/her. Moreover, assume that each 
individual prefers at least four of his/her eight nearest neighbors to be- 
long to same ethnic group as he/she does. Considering himself /herself, 
this implies a small majority of five out nine people in the considered 
neighborhood. If the condition is not met, he or she is assumed to move 
to the nearest site/cell that satisfies his or her desire. Hence, the model 
does not assume optimizing location choice, just satisficing behavior. Nev- 
ertheless, it produces segregation patterns. 



1.1 Migration, Game Theory, and Cooperation 

In game theoretical terms, this movement to a more favorable neighbor- 
hood could be reflected by a higher payoff to persons who moved, called 
"migrants" in the following. Migratory behavior aiming at higher pay- 
offs is called "success-driven motion" |10|11|12] . As we will show in later 
sections, success-driven motion can reproduce residential segregation and 
some other observed phenomena of population dynamics as well, like pop- 
ulation succession (i.e. cycles of changing habitation) [13]. We will also 
study how a change of the spatial population structure can affect the level 
and evolution of cooperation in a population. 

It could be thought that natural selection, implying competition be- 
tween individuals, would result in selflsh rather than cooperative behav- 
ior |14) . Nevertheless, cooperation is widely observed also in competitive 
settings, from bacteria, over animals to humans |15|16] . Game theory 
|17|18|19|20] has been regarded as a powerful framework to investigate 
this problem, as it can be used to quantify the interactions between indi- 
viduals. 

The mathematical description of tendencially selfish behavior is often 
based on the prisoner's dilemma game, in which the "reward" R repre- 
sents the payoff for mutual cooperation, while the payoff for defection 
(cheating) on both sides is reflected by the "punishment" P. Unilateral 
cooperation will incur the so-called "sucker's payoff' S, while the defector 
gains T, the "temptation". Given the inequalities T > R > P > S and 
2R > T + S, defection is tempting and cooperation is risky, so that defec- 
tion is expected to be dominating, although mutual cooperation improves 
the average payoff. 

In a well-mixed population, where an individual interacts with all 
the others, a defector's payoff is always higher than the average payoff, 
which leads to a prosperity of defectors. In an evolutionary setting as 
it is described by the replicator dynamics |l21j, only those strategies that 
generate above-average payoffs have the chance to spread. Therefore, from 
an evolutionary perspective, there will be no cooperators in the end, which 
contradicts the observed cooperation in reality. 

A variety of mechanisms has been proposed to explain the considerable 
level of cooperation observed under certain circumstances. These include 
kin selection, direct reciprocity, indirect reciprocity, group selection and 
network reciprocity [14]. In particular, it is interesting that the spatial 
population structure can signiflcantly change the level of cooperation in a 
population j22|23j . While there could be a co-evolution of social structure 



and cooperation due to migratory behavior, this subject has not been well 
studied in the past. 

1.2 Migration and Population Succession 

Population succession plays an important role in the change of the popula- 
tion structure in urban areas. The term succession is viewed by ecologists 
as an orderly sequence of changes in the composition or structure of a 
biotic community [21]. Later on, it was also used to describe the change 
in neighborhood and urban land use f25]. For example, Cressey [13] an- 
alyzed the movements of immigrants in Chicago from 1898 to 1930 by 
census tracts and provided a detailed picture of the succession processes 
of specific cultural groups. Interestingly, these processes show a cycle of 
invasion, conflict, recession, and reorganization |26| . 

The "invasion" can be initiated by a few "pioneers", who manage 
to enter a new area dominated by "older" residents. For example, some 
rich colored people may move to a white-only neighborhood, where the 
housing price is relatively high. In this stage, discrimination by a housing 
agency could prevent the integration of cultural groups by constraining 
information to minority groups [8J. But nowadays, housing discrimina- 
tion is illegal in many countries, and the hostility toward minority group 
members has declined. Therefore, the possibility of successful entrance 
for those "pioneers" is much larger. 

Following the initial "invaders", invasions can happen on a much 
larger scale, in particular if housing prices go down. Then, previous res- 
idents, who want to sell their houses at a good price, are eventually re- 
placed. People may even move without reasons, just following the trend 
of what the others do, which is a kind of collective herding behavior |i27j . 
Accompanying this replacement process, the changing composition of the 
population may cause conflicts due to incompatible cultural values and 
prejudices [5^. 

Eventually, the population composition is turned over. Although the 
new group can dominate the residential area, it takes time to reorga- 
nize the environment such as churches and to create it according to their 
own requirements. The cycle can repeat as the previous "invaders" be- 
come established residents. Therefore, there is a typical life circle of a 
neighborhood [25], which involves five stages: development, transition, 
downgrading, thinning out, and renewal. If the facilities in a neighbor- 
hood are running down, residents start to seek new places and abandon 
the previous one. In this case, there is usually little conflict. In the sub- 



sequent renewal stage, new people move in at low prices. By renovating 
the houses, the quarter eventually revives. 

1.3 Co-evoultion of Social Structure and Cooperation 

In order to model such intentional movement of individuals, spatial ef- 
fects must be explicitly considered. Spatial games based on lattices [22] 
would allow one to study such effects. In conventional spatial games, in- 
dividuals are uniformly distributed in the simulation area, and change or 
repeat their strategies, following a certain updating rule. For instance, 
an individual is assumed to unconditionally adopt the most successful 
strategy within the neighborhood. On the one hand, this creates typi- 
cal spatio-temporal pattern. On the other hand, spatial structures play 
an important role in the maintenance of cooperation. In the iterative 
prisoner's dilemma, for example, clusters of cooperators are beneficial 
to cooperators [12], while the evolving spatial structure in the snowdrift 
game ^23j, may inhibit cooperation. Conventional spatial games, however, 
neglect the possibility of individuals to move or migrate, although mo- 
bility is a well-known fact of daily social interactions. We will see that 
the movement of individuals, e.g. population succession, can change the 
spatial structure of a population significantly, and influence the evolution 
of cooperation dramatically. 

The above mentioned social processes that are related to the migratory 
behavior of people, can be well integrated into the framework of spatial 
games. Focusing on success-driven motion, we will in the following, study 
the combination of migration, strategic interactions and learning (specif- 
ically imitation). Numerical simulations show that success-driven motion 
is a mechanism that can promote cooperation via self-organized social 
structures. Moreover, very surprisingly, a certain degree of fluctuations 
("noise") can even enhance the level of cooperation. 

2 Spatial Games with Mobility 
2.1 Classification 

Spatial games with mobility ("mobility games") could be classified as 
follows: 

1. Mobility may take place in physical (geographic) space with one, two 
or three dimensions, or it may occur in abstract space (e.g. opinion 
space). Rather than a regular (grid) space, one may also have a (fixed 
or dynamically changing) network structure (e.g. friendship network). 



2. One may distinguish between games with continuous and with dis- 
crete motion. The first ones may be considered as particular cases of 
differential games [2^ and shall be called "motion games". The sec- 
ond ones will be named "migration games" and can be implemented, 
for example, in terms of cellular automata (particularly, if a parallel 
update is performed). 

3. The mobility game may be deterministic or "noisy", i.e. influenced 
by fluctuations. Such "noise" may be introduced by stochastic update 
rules (e.g. a random sequential update or the Fermi rule [23 . 

4. Multiple occupation of a certain location may be possible, allowing for 
the agglomeration of individuals, or it may be prohibited. In the latter 
case, spatial exclusion requires spatial capacity constraints and/or the 
respect or protection of some "private territory" . 

5. In mobility games with spatial exclusion, empty sites are needed. 
Therefore, the density of free locations ("sites") is a relevant model 
parameter. 

6. The frequency, rate, or speed of mobility may be relevant as well and 
should be compared with other time scales, such as the frequency of 
strategy changes, or the average lifetime of a spatial cluster of indi- 
viduals. Depending on the specification of the game, after a transient 
time one may find everything from chaotic patterns [22 upto frozen 
patterns |30|31j . This may be expressed by a viscosity parameter [32j . 

7. Mobility may be random or directed (e.g. "success-driven"). Directed 
mobility may depend on the expected payoff in a certain time point 
(iteration), or it may depend on the cumulative payoff (i.e. the accu- 
mulation of wealth by individuals). 

8. If age, birth and death are considered, one speaks of "demographic 
games" [33] . 

In the following, we will primarily focus on games with success-driven 
migration in two-dimensional geographic space. Furthermore, we will as- 
sume simple strategies (such as all-cooperate or all-defect), no memory 
and no forecasting abilities. 

2.2 Individual Decision Making and Migration 

In "migration games" , each individual / is located at the position Xj of a 
discrete grid and applies a certain strategy i = X ,t) when interacting 
with individuals J at locations Xj in the neighborhood J\f = N{Xi), who 
apply strategies j = j{J, X', t) (see Fig. [T]). The interactions at time t are 
quantified by the payoffs Pij. The overall payoff for individual / resulting 



from interactions with all individuals J in the neighborhood J\f{Xi) at 
time t is 

Pi{t) = Pi{Xi,t)= Yl ^*^('^)- (1) 

j-.XjeAfiXi) 
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Fig. 1. The focal individual / is represented by the empty circle. It can, 
for example, interact with the k = 4 nearest individuals (red) or with 
k = 8 neighbors, which includes four next-nearest neighbors (black). 



We assume that all individuals prefer places that can provide higher 
payoff. However, their movements are restricted by their mobilities and 
the number of free locations, as one can only move to empty places. In our 
model, the mobility is reflected by the migration range M, which could 
be assumed constant (see Fig. [2| or as a function of cumulative payoffs 

t 

Ci{t) = Y[Pj{t')-ci{t')]. (2) 

t'=0 

Here, the cost of each movement c/(t) may be specified proportionally 
to the distance di{t) < M moved: 

ci{t) = (3 dj{t). (3) 

/3 is a constant. 

When the migration range M is restricted by the cumulative payoff, 
one may set 

M{t) = [aCi{t)\, (4) 



where [ J rounds down to the next integer < aC/(t) and a is a constant, 
which is set to 4 in the following. 
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Fig. 2. (A) When M = 1, the focal individual (black circle) can migrate 
within the red square (Moore neighborhood of range 1). (B) Here, the 
migration neighborhood is chosen as Moore neighborhood of range M = 
2. 



The expected payoff for individual / applying strategy z at a free 
location X' within the migration range can be calculated as 

Pi{x',t)= Yl ^^riJ')' (5) 

J':Xj,eAf(X') 

where j' = j' {J' , X' ,t) are the strategies of the individuals J' located 
within the neighborhood M' = J\f{X') centered at X'. Here, we assume 
that an individual can determine this value by testing a neighborhood M' 
by means of some kind of "fictitious play" with its residents. 

All individuals are assumed to maximize the expected payoff by mi- 
grating to the place that promises the highest payoff. Consequently, the 
payoff at the new destination for I is expected to be 

Pf{t + 1) = max Pi{X',t), (6) 
X'eA 

where A represents the area within the migration range M around Xj. 

In our model, individuals decide to move, if the expected payoff at the 
new position Pf{t + 1) satisfies the short-term cost-benefit condition 



Pf{t + 1)-Pi{t)>ci{t). 



(7) 



If we assume that the cost of each movement is smah, and the new 
neighborhood yields a comparatively high cumulative payoff over the time 
period the individual stays at that place, then the cost of movement can 
be neglected. 

Finally, if two or more places promise the same maximum payoff, we 
assume that an individual will choose the closest one. If both, the payoff 
and distance are the same, then the individual will randomly choose one 
of the closest locations in our simulation. Since all the other individuals 
also seek new places that can increase their payoffs, neighborhoods may 
change quickly, and the resulting payoff may fall below the expectations 
before the move (during the fictitious play step). 

2.3 Learning 

Learning allows individuals to adapt their behaviors in response to other 
people's behaviors in an interactive decision making setting. Basic learn- 
ing mechanisms are, for example, unconditional imitation, best reply, and 
reinforcement learning |34] . 

Unconditional imitation means that people copy the behaviors of oth- 
ers. For instance, we can assume that, when i reaches a new place, it im- 
itates the most successful strategy within the neighborhood (if it is more 
successful than the own strategy) , or the most frequently used strategy in 
past interactions. It should be underlined, that it is not easy to identify 
the future strategy of another individual from its displayed past behavior. 
However, in the simplified context of a prisoner's dilemma, where individ- 
uals are assumed to play either all-defect or all-cooperate, one's behavior 
reveals the strategy directly. It may nevertheless change before the next 
iteration due to learning of the neighbors. 

Reinforcement learning is a kind of backward-looking learning behav- 
ior, i.e. people tend to take the actions that yielded high payoffs in the 
past. Assuming that an individual has a set of strategies, among which 
he or she chooses with a certain probability, this probability is increased, 
if the gained payoff exceeds a certain threshold value ("aspiration level") 
[55T56] . 

Best reply is a sophisticated learning model as well. According to 
it, people choose their strategies based on the expectation of what the 
others will do in a way that maximizes their expected payoff. The ability 
to forecast depends on one's belief about the behaviors of the others. 
For instance, one can try to determine the distribution of the neighbors' 
previous actions, and select the own strategy, which replies to it in the 
best way. 



For simplicity, players in our model only interact with the k = 4 
nearest neighbors and adapt their strategies by unconditional imitation. 
Assuming that the updating rules of all the individuals are the same, we 
can specify a simple migration game as follows: 

1. An individual moves to a new position that is located inside the mi- 
gration range M and maximizes the expected payoff. 

2. The individual imitates the most successful strategy within the inter- 
action neighborhood J\f, if it is more successful than the own strategy. 

Note that changing the sequence leads to a different dynamics, and 
the implementation of migration and learning can be varied in many 
ways. Therefore, spatial games with mobility promise to be a rich research 
field. In the following, we will restrict ourselves to some of the simplest 
specifications. 

The above mentioned framework of spatial games with migration spec- 
ifies a kind of cellular automata (CA) model, in which space and time are 
discrete. Furthermore, the set of actions performed in CA models usually 
depends on the last time step only, which allows for high-performance 
computing of large-scale scenarios. CA models have been successfully ap- 
plied to describe a variety of social and physical systems |,33,.37f38j . The 
outcomes of social and physical interactions can be very well integrated 
into them, which can create many interesting dynamics, like the "game 
of life" [39]. 

3 Simulation Results and Discussion 

In the following, we perform a random sequential update of the individ- 
ual (i.e. their migration and learning steps), which is to be distinguished 
from the parallel update assumed in [2^1150] . For a discussion of some 
related advantages, see [^. Our numerical simulations are performed 
on 49x49 grids with 40% empty sites, i.e. a density of 0.6 and periodic 
boundary conditions. 49x49 grids were chosen for better visibility, while 
the statistical evaluations (see Figs. [6] and [7]) were done with 99x99 grids 
for comparability with Nowak's and May's spatial games [22] . In the pris- 
oner's dilemma, the color code is chosen as follows: blue = cooperator, 
red = defector, green = defector who became a cooperator, yellow = co- 
operator who turned into a defector in the last iteration. In other games, 
blue = player of group 1, red = player of group 2, green = player of group 
2 who became a player of group 1 , yellow = player of group 1 who turned 
into a player of group 2. White always corresponds to an empty site. 



3.1 Spontaneous Pattern Formation and Population Structure 

In the migration game, individual preferences are reflected by the pay- 
off matrix, which quantifies the possible outcomes of social interactions. 
Fig. |3] shows a variety of patterns formed when the payoff matrix is mod- 
ified. The outcomes from interactions between individuals are the driving 
forces changing the population structure in space. In the following, we will 
call all the individuals, who apply the same strategy, a "group" g. Group 
1 is represented in blue, group 2 in red. The size Ng of group g is constant 
in true only in the migration-only case without learning. Otherwise, strat- 
egy changes imply changes of individuals between groups and changes in 
group size. We will distinguish the three social relations between groups: 
(i) both groups (or, more exactly speaking, their individuals) like each 
other (Pi2 = i-21 = 1), (ii) group 1 is neutral with respect to group 2, but 
group 2 dislikes group 1 (i-*i2 = 0,P2i = —1), and (iii) intra-group inter- 
actions are more favored than inter-group interactions {Pu = P22 = 1) 
P12 = P21 = 0.5). Intra-group affiliation is always preferred or at least 
neutral (Pu = P22 = 1 or Pu = P22 = 0). As people are allowed to 
move, when they like each other, a strong integration of populations can 
be observed. However, if the affinity is unilateral, then one population 
tends to evade the invasion of the other. Assuming the same mobility in 
both populations, in the current setting of our simulations the chasing 
population gains the majority in the end, if unconditional imitation is 
considered. 

Residential segregation emerges through the interactions of individu- 
als, not only when they dislike each other, but also if individuals within 
the same group like each other more than individual from other groups. 
When inter-group interactions result in smaller payoffs, people attempt 
to maximize their payoff by agglomerating with the same kind of people, 
which eventually results in the segregation of different groups as a side 
effect. 

Some theoretical analysis can be useful to understand the micro-macro 
link. Let Ugi and ng2 be the average number of individuals of group 1 and 
2, respectively, in the interaction neighborhood of an individual using 
strategy i, i.e. belonging to group g = i. Then, the payoff of an individual 
belonging to group g £ {1,2} is 

Pg = nglPgl + ng2Pg2, (S) 

where Pgi is the payoff of an individual of group g when meeting an 
individual using strategy i. 



Migration Only 
M = 5 



(C) 



Fig. 3. Simulation results for 49x49 grids with a density of 0.6 for the 
migration-only case (left), the imitation-only case of conventional spatial 
games (middle) and the combination of imitation with migration (right). 
The color code is chosen as folows: blue = player of group 1, red = player 
of group 2, yellow = player of group 1 who turned into a player of group 
2 in the last iteration, green = player of group 2 who turned into a player 
of group 1 in the last iteration, white = empty site. (A) Pu = P22 = 
0, P12 = P21 = 1- Inter-group interaction is encouraged, which causes 
the integration of two populations. The combination of migration and 
imitation results in the co-existence of both groups and the formation of 
clusters. (B) Pu = P22 = P12 = 1, P21 = —1- When affinity is unilateral, 
one population keeps approaching the other one, which tends to evade. 
The combination of migration and imitation leads to the spreading of 
the chasing group. Co-existence is not observed. (C) Pu = P22 = 1, 
P21 = P21 = 0.5. When interactions between groups are less profitable 
than in the same group, residential segregation occurs as well, but in 
contrast to (B), it stabilizes. 



The total payoffs for individuals of group g is NgPg, while the total 
payoff of both groups is 

r = N1P1 + N2P2, (9) 

where Ng is the number of individuals of group g, i.e. pursuing strategy 
g = i. 

According to success-driven migration, the change of an individual's 
payoff in the noiseless case is always positive, i.e. 

APg = AUglPgl + Au g2 Pg2 > ( 1 0) 

with Augi = ngi{t + 1) - ngi{t) and Ang2 = ng2{t + 1) — ng2{t). 

Assuming P12 ^ Pu and P12 > 0, we will usually have APi ~ 
An\2Pi2 > and Ani2 > 0, which leads to a monotonous increase in 
the number of neighbors of individuals of group 1 . Giving the inequality 
Pii ^ P12 and Pii > 0, we analogously obtain APi k, AnnPn > and 
Ariii > 0. Therefore, when intra-group interactions of group 1 are much 
stronger than inter-group interactions, and positive, clusters of group 1 
will expand, as each migration step will increase the average number of 
neighbors within group 1. However, when Pu < 0, the formation of clus- 
ters of individuals belonging to group 1 by intra-group interactions is 
unlikely. When P12 < 0, an individual of group 1 tends to evade members 
of group 2, which can be regarded as a repulsive effect attempting to keep 
a certain distance between individuals of different groups 

The total change of payoff is given by 

AT' = Ni{AnuPii + AnuPu) + N2{An2iP2i + An22P22) > 0, (11) 

as long as success-driven migration is applied by all the individuals, and 
changes ANi and AN2 in the group sizes are negligible. 

Let us now consider a simple example, where Pu = P22 = 0. Then, 
we have 

AT' = NiAnuPu + N2An2iP2i, (12) 

which reflects the combined effects of interactions between members of 
group 1 and 2. Furthermore, if AT' > 0, we expect that the number 
of neighborships between individuals of both groups will monotonously 
increase. The corresponding macroscopic phenomenon is the spatial inte- 
gration (mixture) of both groups. However, if (P12 -|- P21) < 0, the reduc- 
tion of inter-group links is likely and will result in residential segregation 
(see Fig.[3]). 



Here we have examined how individual preferences can change the 
spatial population structure in a very simple social system considering mi- 
gratory behavior. The revealed social process can, to some extent, reflect 
the dynamics of population succession in urban areas. This corresponds 
to case B in Fig. |3j Consider the payoff matrix in Fig. [3p, and imagine 
that an individual of group 1 happens to be located in the neighborhood 
of group 2. For the previous residents, this may not change a lot. How- 
ever, it attracts other members of group 1, who are not yet living in a 
neighborhood of group 1 individuals. This can trigger collective migration 
of other group 1 members. Since interactions between members of group 
1 and 2 bring positive payoffs only for members of group 1, group 2 will 
finally leave for new places. Of course, this process repeats, just like the 
circle of population succession in Sec. 1.2. 

One may notice that, here, we do not differentiate the intention to 
migrate and the actual migratory behavior. In daily life, however, the 
movement of people is restricted by much more factors such as wealth 
and time, so people do not migrate often, even if they are motivated 
to move. But in our simulations as well, migration activity after a few 
iterations is small: Starting with high migration rate, due to the artificial 
choice of a random initial distribution, the migration rate quickly drops 
to a low level [12] . Of course, other factors determining migration can be 
easily added to the above proposed framework of migration games. 

3.2 Promotion of Cooperation in the Prisoner's Dilemma 

The prisoner's dilemma is an important paradigm for studying the emer- 
gence of cooperation among selfish individuals. Nevertheless, studies of 
the prisoner's dilemma on lattices have not fully explored the effect of 
mobility. It would be logical, of course, for people to evade areas with a 
high level of defection, and to settle down in areas dominated by cooper- 
ators. But would cooperative areas be able to survive invasion attempts 
by defectors or even to spread in space? To answer these questions, we 
will now focus on the effects of success-driven migration on the spatial 
population structure and the level of cooperation. 

Fig. |4] compares the migration-only case with M = 5 (left) with the 
imitation only case corresponding to M = (center) and the combined 
imitation-and-migration case with M = 5 (right). In the imitation-only 
case, the proportion of cooperators is greatly reduced. However, the com- 
bination of migration and imitation strikingly promotes the level of coop- 
eration. Our explanation is that, when individuals have mobility, cooper- 
ative clusters are more likely to be promoted in the presence of invasion 



attempts of defectors. We can see that, in the migration-only case, co- 
operators manage to aggregate and to form clusters. Although defectors 
attempt to enter cooperative clusters, they finally end up at the bound- 
aries of cooperative clusters, as cooper ators split to evade defectors and 
re-aggregate in new places, where defectors are excluded. In the pris- 
oner's dilemma, it is guaranteed that 2R > T -\- S, which means that 
the "attractive force" between cooperators is mutual and strong, while 
the interaction between a cooperator and a defector leads to a unilateral 
attractive force (T > 0). When 5 < 0, a cooperator replies to defectors 
even in a repulsive way. Therefore, defectors are less successful in joining 
or entering cooperative clusters than cooperators are. 

Configurational analysis [12] (see Fig. |5]), indicates that, when P = 
S = 0, and an individual only interacts with k = 4: nearest neighbors, a 
cooperative cluster can turn a defector into cooperator by unconditional 
imitation for T < 1.5R, when the defector is surrounded with one or 
two cooperators. When a defector is surrounded with three cooperators, 
cooperators can resist the invasion of a defector, even if the temptation 
value T is as high as A defector can invade one of the nearby cooper- 
ators, if its neighborhood is fully occupied by cooperators. Therefore, the 
formation of compact clusters is important to support the spreading of 
cooperation. If A; > 4, the spreading of cooperation occurs even for higher 
value of T. 

In spatial games without mobility, the occurrence of a compact coop- 
erative clusters mainly depends on the initial distribution. Giving mobil- 
ities, the migratory behavior can significantly accelerate the formation of 
compact clusters, and promote the level of cooperation. 

Quantitative studies of how migration can promote the level of co- 
operation have to compare the fraction of cooperators in situations with 
mobility and without. Fig. |6] shows the amplification factor, defined as 

S{T,M) = &. (13) 

JO 

f^j is the fraction of cooperators, given the mobility range M and a temp- 
tation value of T. We can see that the level of cooperation in the prisoner's 
dilemma is promoted in a large parameter area, if the punishment P and 
the sucker's payoff are roughly comparable in size. 

However, we have not yet studied the robustness of the migration 
mechanism so far. One may imagine that, once a defector would manage 
to invade a cooperative cluster, it may turn neighboring cooperators into 
defectors as well, thereby eliminating cooperation eventually. While in 




Fig. 4. Simulation results for 49x49 grids with a density of 0.6 for the 
migration-only case (left), the imitation-only case of conventional spatial 
games (middle) and the combination of imitation with migration (right). 
The color code is chosen as follows: blue = player of group 1, red = 
player of group 2, yellow = player of group 1 who turned into a player 
of group 2 in the last iteration, green = player of group 2 who turned 
into a player of group 1 in the last iteration, white = empty site. (A) 
Pii = = 1, = S = -0.2, P21 = T = 1.4, P22 = P = 0. The 
payoff matrix corresponds to a prisoner's dilemma. (B) Pn = i? = 1, 
P12 = S" = 0, P21 = T = 1.4, P22 = P = 0. The sucker's payoff is set 
to zero to be compatible with the payoff matrix studied by Nowak and 
May. (C) Pn= R = 1, P12 = S = -0.2, P21 = T = 1.4, P22 = P = 0. 
The migration and imitation step are inverted here, i.e. an individual first 
imitates, then migrates. (D) Pu = 0.59, P12 = 0.18, P21 = 1, P22 = 0. In 
the snow-drift game, similar structures are found in the migration-only 
and imitation-only case. Giving the possibility to move, frequent switches 
of strategies are observed. See main text for details. 
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Fig. 5. Analysis of the invasion of a cluster of cooperators (black circles) 
by a single defector (cross). For the simplicity of analysis, we assume S = 
P = here, but one can easily generalize the analysis to the situations 
with P > S > 0. (A) Scenario 1: The payoff for the defector is T, while its 
neighboring cooperator obtains a payoff of 3R. Since 3i? > T, the defector 
will become a cooperator. (B) Scenario 2: The payoff for the defector is 
2T, while the maximal payoff among neighboring cooperators is 3R. In 
order to make the defector become a cooperator, we need 3R > 2T, i.e. 
T < 1.5R. (C) Scenario 3: The defector obtains 3T, while the maximal 
payoff of neighboring cooperators is 3R. In order to turn the defector 
into a cooperator, the inequality 3-R > 3T must be satisfied, i.e. T < R. 
This condition can never be met in the prisoner's dilemma. In order that 
cooperators do not copy the defector, 4R > 3T must be satisfied, i.e. 
T < (D) Scenario 4: The payoff for the defector is 4T, the maximal 
payoff of cooperators in the whole community is 4i?. Because of T > 
R, the defector can invade the cooperators nearby. Once a cooperator 
becomes a defector however, the payoff for the defectors will be reduced 
from 4T to 3T, which may stop the further invasion of defectors 



Fig. 6. Amplification factor of the level of cooperation by migration (M = 
5) as a function of the sucker's payoff P12 = S and the punishment P22 = 
P in the prisoner's dilemma {S < P) and the snow drift game {S > P). 
The simulation was performed for 99x99 grids with a density of 0.6. P 
and S were varied between —1 and 1, the payoffs R = 1 and T = 1.3 were 
left fix. 



a noiseless environment, a defector cannot enter the center of a compact 
cooperative cluster, "noise" could make it happen with a certain probabil- 
ity. As defectors in a cooperative cluster can spread (see Fig. |5]D), noise 
could therefore be thought to destroy the enhancement of cooperation by 
success-driven migration. Very surprisingly, this is not the case! 

In order to verify that success-driven migration robustly promotes 
cooperation, we have implemented 3 kinds of noises. In each time step, a 
certain proportion y of individuals was selected to perform the following 
operations after the respective migration and imitation steps: 
Noise 1: The selected players' locations were exchanged with a randomly 
chosen neighboring site ("neighborhood flipping"). 

Noise 2: The selected players' strategies were flipped, i.e. cooperation was 
replaced by defection and vice versa ("strategy flipping"). 
Noise 3: The selected players were removed from the grid, and an equal 
number of players was created at randomly chosen free sites, in order 
to mimic birth and death processes. The newly born players had a 50% 
chance to be cooperators and a 50% chance to be defectors. 

Fig. [7] shows the time evolution of the number of cooperators with 
noise strength y = 2% and y = 10% respectively. Without mobility, 
noise reduces the number of cooperators greatly. For a mobility range 
M = 5, however, we surprisingly find that the level of cooperation can be 
still maintained at a high level. With 2% noise, noises 1 and 3 can even 
increase the number of cooperators compared to the no-noise case! 

In order to understand how noise can promote cooperation to a level 
higher than the level in a noiseless system, we may define a kind of po- 
tential energy function of the system by the negative total payoff: 

E = -T'. (14) 

In a noiseless system, each individual's migration step will increase T, 
and reduce the potential energy of the system, [see Eq. (11)]. Then an in- 
dividual will adopt the most successful strategy within its neighborhood. 
The imitation step and noise can flip the strategies and increase the en- 
ergy of the system. Just as for the energy functions in spin glass models 
|42] , recurrent neural networks [l3] and Boltzman machines , there are 
many local minima of E in the migration game, one of which is reached 
within a few iteration steps. Then the noiseless system behaves station- 
ary. It is stuck in the corresponding meta-stable state, which is highly 
dependent on the initial condition. When noise occurs, it can perturb the 
meta-stable state of the system and drive it to more stable states (see 
Fig. Is]). Only if the noise level is too strong, the system behaves pretty 
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Fig. 7. Time evolution of the number of cooperators. Here, the simulation 
is performed on 99x99 grids with a density of 0.6 and payoffs Pu = R = 1, 
Pi2 = S = 0, P2i = T = 1.3, and P22 = P = 0. 



much in a random way. Therefore, in conclusion, moderate levels of noise 
do not destroy a high level of cooperation — they can even support it. 



Fig. 8. Noise can make the system leave a sub-optimal state and reach 
the globally optimal state. 



Our results for the conventional prisoner's dilemma with P > S con- 
firm the robustness of migration as a mechanism to promote cooperation 
(see Fig. [9]). One can easily see that, without mobility, the proportion of 
cooperators is close to zero, while there is still a certain small number of 



cooperators in the environment without noise. However, when individuals 
can move, cooperation is significantly increased due to the spontaneous 
formation of clusters. Imagine that a defector is located in the center of a 
cooperators' cluster. In the beginning, defection can invade cooperation 
due to the higher payoff (see Fig. |5]D). But once a neighboring cooper ator 
becomes a defector, a defector's payoff is reduced from AT to 3T, if P = 0. 
If more cooperators turn into defectors, the payoff will be further reduced. 
Therefore, the exploitation of cooperators by defectors is self-inhibitory, 
if individuals copy better performing neighbors. Furthermore, a splitting 
of a cooperative cluster can occur, since cooperators try to move to more 
favorable neighborhood as soon as defectors are approaching them. The 
migration of those cooperators can encourage other cooperators, whose 
payoff depends on the mutual interactions, to move as well. That is, de- 
fectors may trigger a collective movement of cooperators. Of course, the 
newly formed cooperative clusters are also likely to be invaded by neigh- 
borhood or strategy flipping, or birth and death process. However, this 
will just repeat the above mentioned migration process. Therefore, coop- 
erators can survive and spread even in a noisy world. Such a dynamical 
change of the population structure through invasion and succession re- 
flects various features of the migratory behavior observed in reality (see 
sec. 1.2). 



In the migration rule studied above, we assume that a favorable neigh- 
borhood can be determined by fictitious play, i.e. some low-cost interac- 
tions with the people in that neighborhood ("neighborhood testing"). One 
may think that this is difficult in reality as it requires to reveal people's 
strategies. However, one may also argue that people tend to migrate to 
high-quality residential areas, which provide better education for children, 
a low crime rate, and other social welfare. In fact, neighborhoods are of- 
ten "labeled" , and it may be assumed that this label (the appearance and 
character of a neighborhood) depends on the total cumulative payoffs (the 
accumulated wealth) of the residents in the neighborhood. Therefore, one 
could assume that individuals try to move to the neighborhood with the 
highest cumulative payoff. The success-driven migration based on such a 
wealth-based "neighborhood tagging" is examined in Fig. 10 Again, we 
find that migration promotes the formation of cooperative clusters and 
an enhanced level of cooperation. 



Imitation Only 
M = 



Migration and Imitation 
M = 5 




Fig. 9. Effect of different kinds of noises on the outcome of the spatial 
prisoner's dilemma with migration but no imitation (left), with imitation, 
but no migration (center), and with both, migration and imitation (right). 
The simulation is performed on 49x49 grids with a density of 0.6. The 
color code is chosen as follows: red = defector, blue = cooperator, green = 
defector who became a cooperator, yellow = cooperator who turned into 
a defector in the last iteration, white = empty site. While the resulting 
level of cooperation is very small in the conventional imitation-only case 
(center), the additional consideration of migration results in large levels of 
cooperation even in the presence of different kinds of noise. (A) No noise. 
(B) Noise 1 (neighborhood flipping). (C) Noise 2 (strategy flipping). (D) 
Noise 3 (birth and death). The payoffs were T = 1.3, R = 1, P = 0.1, 
and S" = in all cases. See main text for details. 




Fig. 10. Migratory prisoner's dilemma with wealth-based neighborhood- 
tagging rather than neighborhood testing as before. The simulation is 
performed on 49 x49 grids with a density of 0.6. The color code is chosen as 
follows: red = defector, blue = cooperator, green = defector who became 
a cooperator, yellow = cooperator who turned into a defector in the last 
iteration, white = empty site. (A) Pu = R = 1, P12 = 5 = 0, P21 = 
T = 1.4, P22 = P = 0. (B) Pn = R= 1, Pi2 = S = 0, P21 = T = 1.4, 
P22 = P = 0, as in (A), but the update rule is inverted, i.e. an individual, 
first imitates, then migrates. In both cases, one can see that, without 
mobility, the proportion of cooperators becomes very low. However, when 
success-driven migration is possible, cooperation can spread, even if the 
sequence of the migration step and the unconditional imitation step is 
inverted. 



4 Conclusions 



We have introduced the concept of migration games by considering success- 
driven motion. Migration games can easily reproduce macroscopic styhzed 
facts of various social phenomena based on individual actions and inter- 
actions. Typical examples are population succession and residential segre- 
gation. These aggregate outcomes emerge from the interactions between 
individuals in a non-trivial way, and a theoretical analysis allows one to 
qualitatively understand the relation between the microscopic interactions 
and the emerging macroscopic phenomena. Nevertheless, further studies 
are required to fully elaborate the micro-macro link in a quantitative way. 

For the prisoner's dilemma, we have shown that self-organized co- 
operative structures can promote the level of cooperation. Moreover, we 
have verified that the enhancement of cooperation by success-driven mo- 
tion is robust to different kinds of noise. Surprisingly, we even find that 
moderate noise levels can promote the cooperation level further. The un- 
derlying mechanism is that, in the migration game, success-driven motion 
will monotonously increase the total payoff in the noiseless system, which 
however can lead the system into a locally optimal state. The effect of 
noise can drive the system out of local optima towards the globally opti- 
mal state. 

The framework of migration games is quite flexible and allows the 
integration of other interactions to study further social processes. 

The spatial structure in our simulation is very simple, and the neigh- 
borhood depends only on an individual's position. Real cities are more 
complex and show a co-evolutionary dynamics. For example, the city 
structure can reflect the distribution of social status groups. Early mod- 
els like the Burgess concentric zone model |l5] divides the city into specific 
areas separated by status rings. The city center is located in the middle 
of the circle, around which newer, higher-quality housing stocks tend to 
emerge at the perimeter of the city. Therefore, the growth of the city 
center will expand adjacent residential zones outwards. 

Status segregation is quite obvious in such an idealized model of city 
structure. Poor people or new immigrants may only afford low quality 
housing. Middle class people live in less compacted neighborhoods. Rich 
people tend to accumulate in particular quarters. 

Burgess's model is based on the bid rent curve. Recognizing that some 
poor people prefer to live near the main transportation arteries and com- 
mercial establishments, Hoyt |l6] modified the concentric zone model to 
take this into account. In Hoyt's model, cities tend to grow in wedge- 



shaped patterns or sectors. Major transportation routes are emanating 
from the central business district (CBD). The residential areas for lower 
income people are located adjacent to the industrial quarters, while upper 
class neighborhoods are far away from industrial pollution and noise. 

It would be natural to extend migration games in order to study the 
co-evolutionary dynamics of population structure and urban growth. On 
the long run, we hope this will contribute to a better understanding and 
planning of the population dynamics in a city. 

Further research work can also study conflicts related with migratory 
behavior, as has been revealed by the empirical research ^47|48j. 
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